One-pass variable bit rate moving pictures encoding

ABSTRACT

A method and apparatus for encoding digital video utilizing a single pass variable bit rate encoding procedure. An encoding quality is set and the bit rate for encoding blocks or frames in the sequence of moving pictures is determined to achieve the selected quality according to the complexity of the video sequence. The bit rate is constrained by predetermined upper and lower bit rate limits.

FIELD OF THE INVENTION

The present invention relates to a method and apparatus for encodingmoving pictures. In particular, the present invention relates to amethod and apparatus for performing variable bit rate control for adigital video encoder.

BACKGROUND OF THE INVENTION

Digital compression has been applied to moving pictures for the purposesof transmissions bandwidth or storage size reduction. One current art ofsuch compression techniques can be derived from the ISO/IEC MPEGStandards, the ISO/IEC 11172-3 (MPEG-1), the ISO/IEC 13818-2 (MPEG-2)and the MPEG-2 TM5 (test model 5), developed by the Moving PictureExperts Group of the International Organisation for Standardization. Thedisclosures of those standards documents are hereby expresslyincorporated into this specification by reference.

In a standard MPEG compliant video encoder, a sequence of movingpictures (e.g. video) is input to the encoder where it is compressedwith a user defined target bitrate. The target bitrate is set accordingto the communication channel bandwidth in which the compressed video isto be transmitted, or the storage media capacity in which the compressedvideo sequence is to be stored.

Several different forms of coding can be employed depending upon thecharacter of the input pictures, referred to as I-pictures, P-pictures,or B-pictures. The I-pictures are intra-coded pictures used mainly forrandom access or scene update. The P-pictures use forward motionpredictive coding with reference to previously coded I- or P-pictures(anchor pictures), and the B-pictures use both forward and backwardmotion predictive/interpolative coding with reference to previouslycoded I- or P-pictures. Furthermore, a group of picture (GOP) is formedin encoded order starting with an I-picture and ending with the picturebefore the next I-picture in the sequence.

The pictures are partitioned into smaller and non-overlapping blocks ofpixel data called Macroblocks (MBs) before encoding. Each MB from a P-or B-picture is subjected to a motion estimation process in whichforward motion vectors, and backward motion vectors in the case of aB-picture MB, are determined using reference pictures from a framebuffer. With the determined motion vectors, motion compensation isperformed where the intra-or inter-picture prediction mode of the MB isfirst determined according to the accuracy of the motion vectors found,followed by generating the necessary predicted MB.

The predicted MB is then subjected to discrete cosine transform (DCT)and DCT coefficients quantization based on quantization matrices (QM)and quantization stepsize (QS). The quantized DCT coefficients of the MBis then run-length encoded with variable length codes (VLC) andmultiplexed with additional information such as selected motion vectors,MB coding modes, quantization stepsize, and/or picture and sequenceinformation, to form the output bitstream.

Local decoding is performed by inverse quantizing the quantized DCTcoefficients, followed by inverse DCT, and motion compensation. Localdecoding is performed such that the reference pictures used in themotion compensation are identical to those to be by an external decoder.

The quantization stepsize (QS) used for quantizing the DCT coefficientsof each MB has direct impact on the number of bits produced at theoutput of the VLC encoding process, and therefore the average output bitrate. It has also a direct impact on the encoding quality, which isoutput picture quality at the corresponding decoder. In general, largerQS generates lower output bit rate and lower encoding quality. In orderto control output bit rate and picture quality so that the resultingbitstream can satisfy channel bandwidth or storage limitations as wellas quality requirements, rate control and quantization controlalgorithms are used.

Some methods for rate control and quantization control can be found inthe abovementioned MPEG-2 TM5 (Test Model 5). These methods comprisegenerally a bit allocation process, a rate control process, and anadaptive quantization process. In the bit allocation process, a targetnumber of bits is assigned for a new picture to be coded according to anumber of previously determined and present parameters. The rate controlstep then calculates a reference quantization stepsize QS_(ref) for eachMB based on the target bits for the picture, the number of bits alreadyused from the target bits in encoding MBs from that picture, and avirtual buffer model as given in MPEG-2 TM5. In the adaptivequantization process, the calculated QS_(ref) is then scaled accordingto local activities of the MB, and an average MB activity determinedfrom the previously coded picture. This scaling is done according to alevel of masking effects of coding noise by human perception for MB withhigh or low activities within a picture. An example of an adaptivequantization technique is disclosed in U.S. Pat. No. 5,650,860, entitled“Adaptive Quantization”. A video buffer verifier (VBV) may also beemployed in such a way that underflow and overflow are prevented asrequired by the MPEG standard to ensure the target bit rate ismaintained. Techniques for underflow detection and protection are alsodisclosed in U.S. Pat. No. 5,650,860.

It is apparent that the fixed target bitrate in the process outlinedabove has little or no relationship to the actual or varying complexityof the video scenes contained in the input picture sequence. The targetbitrate is actually defined by the communication channel bandwidth, orby the target storage capacity for the picture sequence, but theperceptual quality of the resulting pictures when decoded may vary fromgood to annoying from scene to scene according to scene complexity.

For applications where picture sequences are compressed for storage andretrieval, for example DVD (Digital Video Disc or Digital VersatileDisc), variable bit rate (VBR) may be applied on individual segments ofthe picture sequences depending on its scene complexity to maximize bitrate allocation and encoded picture quality. Data bits may be reducedfor less complex scene to save storage space and increase potentialrecording duration of the medium, or the resulting storage saving can beused for coding of more complex scenes.

Similarly, VBR can also be applied to other applications such as a multichannel video broadcasting network. Such channel bandwidth may bedynamically allocated to individual video sequences to be multiplexedtogether so that higher percentage of the bandwidth is used adaptivelyby sequences with complex scenes.

Existing VBR control algorithms such as that disclosed in U.S. Pat. No.5,650,860 require multiple encoding passes to properly distribute databits. In the first coding pass, the bit utilization information isdetermined for each scene or each picture in the input picture sequence.This may be done by fixing the reference quantization stepsize anddisabling the VBV control.

The determined bit utilization information is then be used to generate abit budget for each scene or picture such that an overall target numberof bits to code the sequence is fixed, and so that a maximum bit rate isnot violated. To accomplish this, the bit budget for each picture ismodified so that the VBV buffer does not underflow. In cases thatinitial bit utilization information obtained is unrealistic forgenerating the bit budget, steps from the first coding pass must berepeated with an adjusted reference quantization stepsize. The inputsequence is coded in a final pass using the generated bit budgetinformation to achieve the target bits or bit rate. This form ofmultiple-pass VBR encoder requires very large storage memory for storingthe intermediate bit utilization information, and large computationalcapacity for the additional passes and the bit budget generation.Furthermore, such VBR technique such as this cannot process the inputsequence in real-time.

SUMMARY OF THE INVENTION

An object of the present invention is to provide a one-pass variable bitrate control technique for coding of moving pictures.

In accordance with the invention, there is provided a method forvariable bit rate control in a single pass moving pictures encoder,comprising:

selecting a target picture encoding quality;

selecting upper and lower bit rate limits;

encoding at least one picture based on a target bit rate within theupper and lower bit rate limits;

predicting a current bit rate and an encoding quality based on theresult of the encoding step;

comparing the encoding quality of the at least one encoded picture withthe target picture encoding quality;

adjusting the target bit rate within the upper and lower bit rate limitsaccording to the result of said comparison and the predicted current bitrate, for encoding subsequent pictures; and

repeating, for each picture in a sequence of pictures, said encoding,predicting, comparing and adjusting steps.

In one form of the invention, the picture encoding quality is based on amean square error. Alternatively, the picture encoding quality may bebased on a signal-to-noise ratio. Preferably the comparing stepincluding measuring a difference in picture encoding quality betweencorresponding input and locally decoded pictures.

In a particular form of the invention, the pictures to be encoded arearranged in groups of pictures comprising an I-picture and at least oneP-picture and/or B-picture, and wherein the target bit allocation isadjusted for each picture or plurality of pictures in each group ofpictures. Preferably the target bit allocation is adjusted to achieve atarget bit rate, determined on the basis of said comparison, for eachpicture or plurality of pictures in the group of pictures.

The target picture encoding quality may comprise a target encodingquantization step-size, wherein different target quantization step-sizesare selected for I-, P- and B-pictures.

In one form of the invention, the method may include measuring anaverage quantization step-size for at least one previously encodedpicture, predicting a bit rate for a previously encoded I-, P-, andB-picture, and determining said target bit rate based on said predictedbit rate and a difference between the target encoding quantizationstep-size and the measured average quantization step-size.

The method of the invention may include measuring an average pictureactivity for the moving pictures, and modifying the measured differencein picture encoding quality on the basis of the average pictureactivity.

The present invention also provides a control apparatus for a singlepass moving pictures encoder, comprising:

an input for receiving a target picture encoding quality;

an input for receiving upper and lower bit rate limits; and

a controller for controlling the encoder so as to encode at least onepicture according to a target bit rate within the upper and lower bitrate limits, predict a current bit rate and an encoding quality based onthe result of the encoding, compare the encoding quality of the at leastone encoded picture with the target picture encoding quality, and adjustthe target bit rate according to the result of said comparison and thepredicted current bit rate for encoding subsequent pictures.

Preferably the moving picture encoder includes a frequency transformcoefficient quantizer for quantization of the encoded picture data, andwherein the controller comprises a bit rate controller coupled tocontrol the quantization step size of the quantizer, a quantization stepsize comparator for comparing, as a measure of encoding quality, andactual quantization step size with a target quantization step size basedon the target picture encoding quality, a bit allocation processorcoupled to control the bit rate controller according to a number of bitsremaining for encoding a group of pictures, and a target bit rateestimator coupled to receive the upper and lower bit rate limits andcoupled to the bit allocation processor and the bit rate controller forcontrolling the quantization so that the required bit rate for thequantized picture data is within the upper and lower bit rate limits.

The present invention further provides a single pass variable bit ratevideo picture encoder comprising:

a picture input for receiving data for a plurality of moving pictures;

a target quality input for receiving a target quality measure forencoded pictures;

an encoder output for supplying encoded picture data;

a bit rate limit input for receiving upper and lower bit rate limits forthe encoded picture data;

a bit rate predictor for predicting a current bit rate;

a frequency transform processor for frequency transform encoding picturedata from the picture input;

a coefficient quantize for quantizing the encoded picture data accordingto a quantization step size;

an encoding quality estimator for measuring an encoding quality ofquantized encoded pictures; and

a bit rate controller for dynamically controlling the quantization stepsize of the coefficient quantize based on the predicted current bit rateand a comparison of the target quality and the measured quality so thatthe encoder output remains within the upper and lower bit rate limits.

The video picture encoder may further include a frame bit counter for anumber of remaining bits available for encoding a group of pictures, anda bit allocation processor for controlling the bit rate controlleraccording to the remaining available bits.

The video picture encoder may further include a quality comparator forcomparing the target quality with a measured encoding quality, andwherein the bit rate controller controls the quantization step sizebased on a difference between the target and measured qualities.

In one form of the encoder, the measured encoding quality comprises thequantization step size.

The video picture encoder may further include a local decoder fordecoding the quantized encoded picture data, and a quality measurementprocessor for determining a quality difference between correspondinginput and locally decoded pictures.

In another form of the encoder, the quality measurement processordetermines a difference in signal-to-noise ratio. Alternatively, thequality measurement processor may determine a mean square error.

It is possible in most applications that only a maximum bit rate andoptionally a minimum bit rate are specified. Such applications mayinclude a randomly accessible recording medium or packetizedcommunication network with variable instantaneous bit rate but also amaximum bandwidth specification. In addition, such applications may alsorequire that the input must be compressed in real-time, for example,live broadcasting or live recording cannot make use of multi-passencoding. Therefore, it is also an object of the present invention toprovide an encoder which can be operated in real time in a variable bitrate mode within the maximum and minimum bitrate boundary of the targetapplication.

While encoding an input moving picture sequence, the present inventioncontinuously measures the resulting encoding picture quality, comparesit to a defined target quality, and adjusts the encoding bit rateaccordingly. By varying the target bit rate of the encoder within adefined maximum bit rate and a minimum bit rate according to a definedtarget encoded picture quality and the scene complexity, the encoderensures consistent picture quality when possible and also that themaximum and minimum bit rate of the target application are not violated.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention is described in greater detail hereinafter, by way ofexample only, with reference to the accompanying drawings, wherein:

FIG. 1 is a block diagram of an MPEG compliant constant bit rate videoencoder constructed according to the prior art;

FIG. 2 is a block diagram of a variable bit rate moving pictures encoderaccording to one embodiment of the present invention; and

FIG. 3 is a block diagram of a moving pictures encoder according to analternate embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

A block diagram of a generalized MPEG compliant video encoder 150 isillustrated in FIG. 1. A sequence of moving pictures (e.g. video) isinput at 100 to the encoder 150 where it is compressed with a userdefined target bitrate provided at 111. The target bitrate is setaccording to the communication channel bandwidth in which the compressedvideo is to be transmitted, or to the storage media capacity in which atarget duration of the compressed video is to be stored.

Input pictures, which may be pre-processed for noise reduction or otherpurposes, are labelled as either I-pictures, P-pictures, or B-pictures,and stored in a frame buffer 101 according to re-ordering and processingneeds. The I-pictures are intra-coded pictures used mainly for randomaccess or scene update. The P-pictures use forward motion predictivecoding with reference to previously coded I- or P-pictures (anchorpictures), and the B-pictures use both forward and backward motionpredictive/interpolative coding with reference to previously coded I- orP-pictures. Furthermore, a group of pictures (GOP) is formed in encodedorder starting with an I-picture and ending with the picture immediatelybefore the next I-picture in the sequence. In the case of an MPEG-2encoder, the input picture sequence may be either a field or framestructure sequence derived for an interlaced or progressive source.

All pictures are partitioned into smaller and non-overlapping blocks ofpixel data called Macroblocks (MBs) before actual encoding. Each MB froma P- or B-picture is subjected to a motion estimation process 102 inwhich forward motion vector(s), ad backward motion vector(s) for thecase of a B-picture MB, are determined using reference picture(s) fromthe frame buffer 101. With the determined motion vector(s), motioncompensation 103 is performed where the intra- or inter-pictureprediction mode of the MB is first determined according to the accuracyof the motion vector(s) found, followed by generating the necessarypredicted MB. For the case of an I-picture MB or intra-mode MB, thepredicted MB is the input MB (i.e. no prediction).

The predicted MB is subjected to discrete cosine transform (DCT) 104 andDCT coefficient quantization 105 based on quantization matrices (QM) andquantization stepsize (QS). Note that for MPEG-2, field or frame basedDCT can be selected in 104. The quantized DCT coefficients of the MB isthen run-length encoded with variable length codes (VLC) at 106 andmultiplexed with all necessary side information, for example selectedmotion vectors, MB coding modes, quantization on stepsize, picture andsequence information, to form the output bitstream. The bitstream isstored in an encoder output buffer 107 for output at 108.

Local decoding is performed by inverse quantizing the quantized DCTcoefficients (109), followed by inverse DCT 110, necessary motioncompensation 103, and finally storing the reconstructed MB into framebuffer 101. Local decoding is performed such that the reference picturesused by motion compensation 103 is identical to those to be used by anexternal decoder.

The quantization stepsize (QS) used in 118 for quantizing the DCTcoefficients of each MB has a direct impact on the number of bitsproduced at the output of the run-length VLC encoding process 106, andtherefore the average output bit rate. It has also a direct impact onthe encoding quality, which is output picture quality at thecorresponding decoder. In general, larger QS generates lower output bitrate and lower encoding quality. In order to control output bit rate andpicture quality so that the resulting bitstream can satisfy channelbandwidth or storage limitations as well as quality requirements, ratecontrol and quantization control algorithms are used.

Previously used methods for rate control and quantization control can befound in the MPEG-2 TM5 (Test Model 5), which can be summarized in threesteps. Step one is the bit allocation process 112 which assigns a targetnumber of bits (T_(I), T_(P), or T_(B) respectively) for a new I-, P- orB-picture to be coded. The target number of bits can be computed asfollows:$T_{I} = {\max \lbrack {\frac{R}{1 + \frac{N_{P}X_{P}}{X_{I}K_{P}} + \frac{N_{B}X_{B}}{X_{I}K_{B}}},\frac{Bit\_ Rate}{8 \times {Picture\_ Rate}}} \rbrack}$$T_{P} = {\max \lbrack {\frac{R}{N_{P} + \frac{N_{B}K_{P}X_{B}}{K_{B}X_{P}}},\frac{Bit\_ Rate}{8 \times {Picture\_ Rate}}} \rbrack}$$T_{B} = {\max \lbrack {\frac{R}{N_{B} + \frac{N_{P}K_{B}X_{P}}{K_{P}X_{B}}},\frac{Bit\_ Rate}{8 \times {Picture\_ Rate}}} \rbrack}$

where

X_(I)=S_(I)Q_(I), X_(P)=S_(P)Q_(P), X_(B)=S_(B),Q_(B),

S_(I), S_(P), S_(B) are number of bits generated by previously encodedI-, P-, B-picture respectively as computed by block 114,

Q_(I), Q_(P), Q_(B) are the average QS used by previously encoded I-,P-, B-picture respectively as computed by block 113,

Bit_Rate is the Target Bitrate supplied at 111,

Picture_Rate is number of pictures coded per second,

K_(P) and K_(B) are constants (e.g. 1.0 and 1.4 respectively),

N_(P) and N_(B) are the number of P- and B-pictures remaining in thecurrent GOP.

R is the remaining number of bits assigned to the GOP, where R isupdated as follows:

After encoding a picture, R=R−S_(I,P,B)

where S_(I,P,B) is the number of bits generated in the picture justencoded (picture type is I, P, or B)

Before encoding the first picture in a GOP (an I-picture):

R=G+R

G=Bit_Rate*N/Picture_Rate

N is the number of pictures in the GOP.

In the second step, rate control (115) is performed. Basically, thisstep involves calculation of a reference quantization stepsize QS_(ref)for each MB based on the target bits (T_(I), T_(P), or T_(B)) for thepicture, the accumulated number of bits used from the target bits, and avirtual buffer model as given in MPEG-2 TM5.

Adaptive quantisation 118 is used in the third step for quantizationcontrol. In this step, the calculated QS_(ref) is scaled according tothe local activities of the MB as generated by an MB activitycalculation process 116, and the average MB activity of the previouslycoded picture (or optionally current picture) as produced by a frameactivity average process 117. This scaling is done according to a levelof masking effects of coding noise by human perception for MBs with highor low activities within a picture. As previously mentioned, an exampleof an adaptive quantization technique can be found in U.S. Pat. No.5,650,860, “Adaptive Quantization”.

The frame bit-count module 114 may contain a video buffer verifier (VBV)which is a virtual model of an input buffer of an external decoder. TheVBV is maintained in such a way that underflow and overflow areprevented as required by the MPEG standard to ensure target bit rate 111is maintained. Techniques for underflow detection and protection mayalso be found in U.S. Pat. No. 5,650,860.

As mentioned, this form of encoding does not take account of the actualor varying complexity of the video scenes contained in the input picturesequence 100. Consequently, the perceptual quality of the resultingpictures may vary from good to annoying from scene to scene according toscene complexity. Variable bit rate (VBR) coding may be therefore beapplied on individual segments of the picture sequences depending on itsscene complexity to maximize bit rate allocation and encoded picturequality, and the bandwidth saving used for coding of more complexscenes, for example. However, existing VBR control algorithms such asthat disclosed in U.S. Pat. No. 5,650,860 require multiple encodingpasses to properly distribute data bits. Multiple-pass VBR encoderrequire huge storage memory for storing the intermediate bit utilizationinformation, and large computation needs for the additional passes andthe bit budgets generation, and furthermore such techniques cannotprocess the input sequence in real-time.

Before encoding an input picture sequence, an encoder according to anembodiment of the present invention is set with maximum and minimum bitrate of target application (the minimum bit rate for some applicationsmay be zero), and a target quality for encoded pictures. An initialtarget bit rate will then be set or generated. The encoder compressesthe input sequence according to this target bit rate using conventionalbit rate control method.

After encoding a picture or a certain number of pictures, the encodingquality (quality of encoded pictures) is measured against the target. Ifthe encoding quality is lower than the target quality, the targetencoding bit rate will be increased. On the other hand, if the encodingquality is higher than the target quality, the target bit rate will bedecreased. The level of new target bit rate adjustment may depend on thecurrent target bit rate and the predicted bit rate based on encodinghistory so that it is not over adjusted unnecessarily. The new targetbit rate is also checked against the defined maximum and minimum bitrate.

The encoder will then re-adjust its rate control system with the newtarget bit rate, and continue to encode the rest of the input sequence.Furthermore, the encoder can ensure VBV compliant with existing methodsusing the defined maximum bit rate and/or the target bit rate. Theprocess of determining new target bit rate based on encoding quality ofpast encoded picture(s) is repeated as needed.

In FIG. 2 a block diagram of a one-pass variable bit rate movingpictures encoder 250 according to one embodiment of the presentinvention is illustrated. Those of ordinary skill in the art willrecognise that a number of the functional blocks illustrated are commonwith the conventional MPEG encoder described above. The input movingpicture sequence 200 is processed picture by picture according to itspossible assigned picture coding type (I-, P-, of B-picture). A framebuffer 201 is used to temporarily store the input pictures according tore-ordering and processing needs. Each input picture is divided intoMacroblocks (MB) before encoding. Temporal redundancy of each MB may beremoved by motion estimation 202 and motion compensation 203.

After necessary motion compensation, a MB is subjected to discretecosine transform (DCT) 204 and DCT coefficient quantization 205 based onquantization matrices and quantization stepsize. The quantized MB of I-or P-picture is inverse quantized 209, inverse discrete cosinetransformed 210, and subjected to corresponding motion compensation 203so that the quantized MB is reconstructed and stored locally in framebuffer 201 for future motion estimation and compensation needs. Thequantized MB is run-length encoded with VLC (variable length coding) at206, together with all necessary side information to form the encodedbitstream of the input sequence. The encoded bitstream is stored in anoutput buffer 207 of the encoder for output at 208 at desired datarates.

Methods of motion estimation 202, motion compensation 203, DCT 204, DCTcoefficient quantization 205, run-length VLC encoding 206, inversequantization 209, and inverse DCT 210 may be those defined and/orallowed in the ISO/IEC MPEG-1 or MPEG-2 standards.

Before encoding, a target cuality of encoding (219) and maximum/minimumbit rate (211) are set. The maximum and minimum bit rates (BR_(max) andBR_(min)), given as bits per second, define the boundary bit rates whichthe encoder shall operate at, and the target quality defines the encodedpicture quality that the encoder shall target by constantly adjusting atarget bit rate within the given bit rate boundaries. The minimum bitrate may be set to zero.

Initially, target bit rate estimation 223 may be performed based the settarget quality and the max/min bit rates to generate an initial targetbit rate (BR_(target)). Such initial estimate may be based onexperimentally determined fixed values which may not directly relate tothe picture(s) to be coded. With the target bit rate, the bit allocation212 is performed to determine a target number of bits for the picture tobe coded (T_(I,P,B)). A rate controller 215 determines a referencequantization step size (QS_(ref)) for each MB based on the estimatednumber of bits for the picture (T_(I,B,P)) and the bit utilization bythe run-length VLC encoder 206. The QS_(ref) of each MB in the pictureto be coded can be computed, for example, as follows:${QS}_{ref} = \frac{D_{I,P,B} \times K_{l}}{( {2 \times {{BR}_{target}/{picture\_ rate}}} )}$

where

D_(I,P,B) is virtual buffer fullness of corresponding I-, P-, orB-picture, updated (after coding each MB) by the difference between thebits used by the MB and the bits allocated to the MB based on thecorresponding T_(I,B,P),

a set of initial values for D_(I), D_(P), D_(B) may be assumed atsequence start, and

K_(I) is a constant (eg. 31).

Adaptive quantization 218 may be applied in which the determinedQS_(ref) is scaled according to the local activities of the MB asgenerated by MB activity calculation process 216 and the average MBactivity of the previously coded (or optionally current picture) asproduced by the frame activity average process 217. Exampleimplementations of the MB activity calculation 216, frame activityaverage 217, and adaptive quantization process 218 are found in MPEG-2TM5. The output quantization stepsize (QS) is used to quantized DCTcoefficients of the MB.

After encoding a picture, the number of bits (S_(I,P,B), correspondingvalue for I-, P-, or B-picture) generated by encoding the picture isaccumulated by Frame Bit-Count 214, and the quantization stepsize (QS)is averaged by Frame Q Step Average 213 process. The encoded picturequality is also determined by Frame Quality Measure 220 process. Onemethod of determining the encoded picture quality is by the averagevalue of the reference quantization stepsize (QS_(average)) used forcoding the picture since it indicates roughly the amount of quantizationnoise in the encoded picture. In this method, the target quality set at219 is defined as the target reference quantization stepsize(QS_(target)). Three different QS_(target) values may be set at 219 forcorresponding I-, P-, and B-pictures; alternatively, the values can bedetermined by:

QS_(target) for I-pictures=QS_(target)

QS_(target) for P-pictures=K_(P)*QS_(target)

QS_(target) for B-pictures=K_(B)*QS_(target)

where K_(P) and K_(B) are constants which can be experimentallydetermined (example K_(P)=1.0 and K_(B)=1.4).

A further condition may be applied to the final QS_(target) forB-pictures such that it is not lower than the QS_(average) of the lastcoded I- or P-picture so that bits can be efficiently used to improvethe quality of anchor I- or P-pictures first before improving thequality of the B-pictures.

In the process of Frame Quality Measure 220 process, the target qualityQS_(target) is compared to the picture quality QS_(average). Since ahigher value of QS_(average) implies higher quantization noise andtherefore lower encoded picture quality, when the value of QS_(average)is found to be higher than QS_(target), the difference of the two valueswill be used to increase the target bit rate (BR_(target)) at targetbitrate estimator 222. On the other hand, when QS_(average) is lowerthan QS_(target), BR_(target) is decreased at target bitrate estimator222. Optionally, the value of QS_(target) may be used at the ratecontroller 215 as a lower limit for the final output QS_(ref) value sothat when the target is reached (target quality reached), bits are savedfor future encoding immediately. Hence, the output referencequantization stepsize at rate controller 215 may be set according to:

if (QS_(ref)<QS_(target)), then QS_(ref)=QS_(target).

For target bit rate estimation, a rate-quantization model for exampleone developed by Wei Ding and Bede Liu, “Rate Control of MPEG VideoCoding and Recording by Rate-Quantization Modelling”, IEEE Trans. onCircuit and Systems for Video Technology, Vol. 6, No. 1, February 1996,may be adopted. To avoid large estimation error or complex local fittingof the rate-quantization model, an alternative embodiment of the presentinvention may consist of a method of target bitrate estimationcomprising the steps of predicting a bitrate (BR_(predicted)) at aBitrate Predictor 221 based on bits used for encoding the last I-, P-,and B-picture, and estimating a new target bitrate (BR_(target)) at thetarget bitrate estimator 222 based on the said predicted bit rate andthe difference between QS_(average) and QS_(target). The predictedbitrate (BR_(predicted)) and the new target bitrate (BR_(target)) may becomputed as follows:${BR}_{predicted} = \frac{( {S_{I} + {n_{P} \times S_{P}} + {n_{B} \times S_{B}}} )}{( {1 + n_{P} + n_{B}} )}$${BR}_{target} = {{BR}_{predicted} + {K_{2} \times \frac{( {{QS}_{average} - {QS}_{target}} )}{{QS}_{average}}}}$

 if (BR_(target)>BR_(max)), then BR_(target)=BR_(max)

if (BR_(target)<BR_(min)), then BR_(target)=BR_(min)

where

S_(I), S_(P), S_(B) are number of bits generated by previously encodedI-, P-, B-picture,

n_(P) is the total number of P-pictures in the current GOP,

n_(B) is the total number of B-pictures in the current GOP, and

K₂ may be a constant, or a factor of BR_(predicted), BR_(max), orBR_(target).

The new target bit rate (BR_(target)) can be computed after encodingeach picture or after encoding a certain number of pictures. The bitallocation process 212 and the rate controller 215 are updated with thenew BR_(target) value once it is determined. An embodiment of the bitallocation process 212 comprises the steps of:

a) Before encoding the first picture in a group of pictures (GOP),determining R which is the remaining number of bits assigned to the GOP:${R+={\frac{N}{picture\_ rate} \times {BR}_{target}}},{{{then}\quad {set}\quad {BR}_{old\_ target}} = {BR}_{target}}$

 where:

N is the number of pictures in the GOP

R=0 before start of sequence, and

+=is the accumulate function

b) Otherwise, if the picture to be coded is not the first picture of aGOP and a new target bit rate (BR_(target)) is determined, then updatingR as given by:${R+={\frac{( {N_{P} + N_{B}} )}{picture\_ rate} \times ( {{BR}_{target} - {BR}_{old\_ target}} )}},{{{then}\quad {set}\quad {BR}_{old\_ target}} = {BR}_{target}}$

 where:

N_(P) and N_(B) are the number of P- and B-pictures remaining in thecurrent GOP

c) Obtaining the average quantization stepsize (Q_(I), Q_(P), or Q_(B))from previously encoded pictures (I-, P-, or B-picture respectively) asdetermined by Frame Q Step average process 213,

d) Obtaining from frame bit-count 214 the value of the number of bits(S_(I), S_(P), or S_(B)) used to encode the past I-, P-, or B-pictures,and removing number of bits used by the last coded picture (S_(I,P,B))from R, hence: $\begin{matrix}{R-=S_{I}} & {{if}\quad {last}\quad {coded}\quad {picture}\quad {is}\quad {an}\quad I\text{-}{picture}} \\{R-=S_{P}} & {{if}\quad {last}\quad {coded}\quad {picture}\quad {is}\quad {an}\quad P\text{-}{picture}} \\{R-=S_{B}} & {{if}\quad {last}\quad {coded}\quad {picture}\quad {is}\quad {an}\quad B\text{-}{picture}}\end{matrix}$

e) Finally, determining the target number of bits (T_(I), T_(P), orT_(B)) to be used for coding the current picture. T_(I), T_(P), or T_(B)for respective I-, P-, or B-picture are given, for example, by:$T_{I} = \frac{R}{1 + \frac{N_{P}X_{P}}{X_{I}K_{P}} + \frac{N_{B}X_{B}}{X_{I}K_{B}}}$$T_{P} = \frac{R}{N_{P} + \frac{N_{B}K_{P}X_{B}}{K_{B}X_{P}}}$$T_{B} = \frac{R}{N_{B} + \frac{N_{P}K_{B}X_{P}}{K_{P}X_{B}}}$

 where

X_(I)=S_(I)Q_(I), X_(P)=S_(P)Q_(P), X_(B)=S_(B),Q_(B),

K_(P) and K_(B) are constants.

A minimum value (T_(min)) may be set as the lower boundary for T_(I),T_(P), T_(B). For example:${T_{\min} = \frac{{BR}_{target}}{K_{3} \times {Picture\_ Rate}}},$

FIG. 3 is a block diagram illustrating another embodiment of a one-passvariable bit rate moving pictures encoder 350 according to the presentinvention. In this embodiment, the encoding quality 324 is determined bycomparing the original input picture taken from input 300 and thelocally decoded picture 325. Conventional methods such as the meansquare error (MSE) or the signal-to-noise ratio (SNR) measures can beused as comparison. This measure is further compared at Frame QualityMeasure 320 with the definable Target Quality 319. When the encodingquality is lower than the target quality, the target bitrate estimator322 will set the target bitrate higher, on the other hand, if theencoding quality is higher than the target quality, the target bitrateis set lower.

An input picture or a given number of input pictures are first encodedwith an initial set of encoding parameters. Each of these pictures maybe given a picture coding type of either I-, P-, or B-picture. Theencoding technique such as those defined by the MPEG-1 or MPEG-2standards may be utilized. Such techniques include motion estimation302, motion compensation 303, discrete cosine transform 304, DCTcoefficient quantization 305, and run-length VLC encoding 306 fortemporal, spatial, and statistical redundancy reduction. A frame buffer301 is used for storing the input pictures for re-ordering andprocessing needs.

Before each picture is coded, its target amount of bits is set by bitallocation process 312 based on the initial set of coding parameters aswell as any updated information from Frame Q Step averaging process 313and Frame bit-count 314. Rate Controller 315 determines referencequantization stepsize for each MB while coding the picture. Conventionaladaptive quantization 318 which depends on MB activity calculation 316and frame activity average 317 can be applied to each referencequantization stepsize to generate final quantization stepsize for theDCT Coeff quantizer 305.

The coded pictures are locally decoded by inverse quantization 309,inverse discrete cosine transform 310, inverse of the motioncompensation 303 to form the locally decoded pictures 325. It is storedin the frame buffer 301 if necessary for future prediction use (ie.anchor pictures). The locally decoded pictures 325 are used withreference to the original input pictures 300 for encoding qualitydetermination 324. The determined encoding quality is compared to thetarget quality given by 319 at frame quality measure process 320. It isalso possible to offset the determined encoding quality by the pictureactivity (ACT_(average)) before the comparison so as to take intoaccount the masking effects of the human visual system to the averagelevel of activity within the picture. For example if the encodingquality is determined by the peak signal-to-noise ratio, then theresulting encoding quality difference (SNR_(diff)) at the output offrame quality measure 320 can be given by:

SNR_(diff)=SNR_(target)−(SNR_(picture)+K₅×ACT_(average))

where

SNR_(picture) is the output of encoding quality measure 324,

SNR_(target) is the target quality definable at 319,

K₅ is a constant (eg. 0.018), and

ACT_(average) is the average MB luminance block variance.

One method of adjusting BR_(target) based on SNR_(diff) at the targetbitrate estimator 322 is given by:

BR_(target)=BR_(predicted)+K₆×BR_(current)×SNR_(diff)

if (BR_(target)>BR_(max)), then BR_(target)=BR_(max)

if (BR_(target)<BR_(min)), then BR_(target)=BR_(min)

where

BR_(predicted) is the result of bitrate predictor 321,

BR_(current) is the previous value of BR_(target) stored at currentbitrate 323 memory,

K₆ is a suitable constant,

BR_(max) and BR_(min) are the maximum and minimum bitrate set at 311.

The bit allocation module 312 and the rate controller 315 take theupdated BR_(target) from the target bitrate estimator 322 and performnecessary bit allocation and bit rate control such that the resultingcompressed moving picture bitstreams will have encoded bit rate close toBR_(target). Existing techniques of VBV (video buffer verifier)underflow detection and protection may be applied using BR_(max) as thereference bit rate to ensure the maximum bitrate of the output bitstreamis not violated.

Because the target bit rate BR_(target) for encoding is adapted based onfeedback on encoding quality, the encoding quality will always convergeto a defined target quality. In such a way the encoding accommodatesvariable bit rates in real-time within single computational pass, andthe encoding process can be better tuned to changes in moving picturesscene content complexity. The overall perceptual quality of the encodedmoving pictures sequence will be relatively consistent with reference tothe defined target.

Embodiments according to the present invention can accommodatesimplified models for estimation of target bit rate based on feedbackand prediction. Overall impact to implementation and computationcomplexity as well as memory storage requirements of the encoder isminimized.

The foregoing detailed description of embodiments of the invention hasbeen presented by way of example only, and is not intended to beconsidered limiting to the invention as defined in the claims appendedhereto.

What is claimed is:
 1. A method for variable bit rate control in asingle pass moving pictures encoder, comprising: selecting a targetpicture encoding quality; selecting upper and lower bit rate limits;encoding at least one picture based on a target bit rate within theupper and lower bit rate limits; predicting a current bit rate and anencoding quality based on the result of the encoding step; comparing theencoding quality of the at least one encoded picture with the targetpicture encoding quality; adjusting the target bit rate within the upperand lower bit rate limits according to the result of said comparison andthe predicted current bit rate, for encoding subsequent pictures; andrepeating, for each picture in a sequence of pictures, said encoding,predicting, comparing and adjusting steps.
 2. A method as claimed inclaim 1, wherein the step of encoding at least one picture based on thetarget bit rate includes determining a target bit allocation for the atleast one picture using the target bit rate.
 3. A method as claimed inclaim 2, wherein the pictures to be encoded are arranged in groups ofpictures comprising an I-picture and at least one P-picture and/orB-picture, and wherein the target bit allocation is adjusted for eachpicture or plurality of pictures in each group of pictures.
 4. A methodas claimed in claim 1, 2 or 3, wherein the picture encoding quality isbased on encoding quantization step-size.
 5. A method as claimed inclaim 1, 2 or 3, wherein the picture encoding quality is based on a meansquare error.
 6. A method as claimed in claim 1, 2 or 3, wherein thepicture encoding quality is based on a signal-to-noise ratio.
 7. Amethod as claimed in claim 5 or 6, wherein the comparing step includesmeasuring a difference in picture encoding quality between correspondinginput and locally decoded pictures.
 8. A method as claimed in claim 7,including measuring an average picture activity for the moving pictures,and modifying the measured difference in picture encoding quality on thebasis of the average picture activity.
 9. A method as claimed in claim8, including measuring a signal-to-noise ratio for at least onepreviously encoded picture, and determining said target bit rate basedon said predicted current bit rate and on a difference between themeasured signal-to-noise ratio and a target signal-to-noise ratio assaid target picture encoding quality.
 10. A method as claimed in claim9, wherein the predicted current bit rate and the target bit rate aredetermined according to:SNR_(diff)=SNR_(target)−(SNR_(picture)+K₅×ACT_(average))${BR}_{predicted} = \frac{( {S_{I} + {n_{P} \times S_{P}} + {n_{B} \times S_{B}}} )}{( {1 + n_{P} + n_{B}} )}$

 BR_(target)=BR_(predicted)+K₆×BR_(current)×SNR_(diff) where:SNR_(picture) is the measured signal-to-noise ratio, SNR_(target) is thetarget signal-to-noise ratio, ACT_(average) is the measured averagepixel activity, K₅ and K₆ are constants, SNR_(diff) is the measureddifference in picture encoding quality, BR_(predicted) is the predictedcurrent bit rate, S_(I), S_(P), and S_(B) are number of bits generatedby said previously encoded I-, P-, B-pictures, respectively n_(P) is atotal number of P-pictures in the group of pictures, n_(B) is a totalnumber of B-pictures in the group of pictures, BR_(current) is apreviously determined BR_(target), and BR_(target) is said target bitrate.
 11. A method as claimed in claim 4, wherein said target pictureencoding quality is based on a target encoding quantization step-size.12. A method as claimed in claim 11, wherein different targetquantization step-sizes are selected for I-, P-, and B-pictures.
 13. Amethod as claimed in claim 11 or 12, including measuring an averagequantization step-size for at least one previously encoded picture, anddetermining said target bit rate based on said predicted current bitrate and a difference between the target encoding quantization step-sizeand the measured average quantization step-size.
 14. A method as claimedin claim 13, wherein maximum and minimum bit rates are selectedcorresponding to the upper and lower bit rate limits, and wherein thepredicted current bit rate and the target bit rate are determinedaccording to:${BR}_{predicted} = \frac{( {S_{I} + {n_{P} \times S_{P}} + {n_{B} \times S_{B}}} )}{( {1 + n_{P} + n_{B}} )}$${BR}_{target} = {{BR}_{predicted} + {K_{2} \times \frac{( {{QS}_{average} - {QS}_{target}} )}{{QS}_{average}}}}$

 if (BR_(target)>BR_(max)), then BR_(target)=BR_(max) if(BR_(target)<BR_(min)), then BR_(target)=BR_(min) where BR_(predicted)is said predicted current bit rate, BR_(target) is said target bit rate,BR_(max) is said maximum bit rate, BR_(min) is said minimum bit rate,S_(I), S_(P), S_(B) are number of bits generated by said previouslyencoded I-, P-, B-pictures, respectively, n_(P) is a total number ofP-pictures in the group of pictures, n_(B) is a total number ofB-pictures in the group of pictures, and K₂ is a constant or a factor ofBR_(predicted), BR_(max), or previous BR_(target).
 15. A method asclaimed in claim 10 or 14, including determining a number of bits Rremaining for encoding the group of pictures comprising the steps of:before encoding a first picture in a said group of pictures, determiningR according to${R+={\frac{N}{picture\_ rate} \times {BR}_{target}}},{{{then}\quad {set}\quad {BR}_{old\_ target}} = {BR}_{target}}$

where: N is the number of pictures in the group of pictures,picture_rate is the number of pictures encoded per unit time, andBR_(target) is said target bitrate, and before encoding each subsequentpicture in the group of pictures, determining R according to${R+={\frac{( {N_{P} + N_{B}} )}{picture\_ rate} \times ( {{BR}_{target} - {BR}_{old\_ target}} )}},{- S},{{{then}\quad {set}\quad {BR}_{old\_ target}} = {BR}_{target}}$

where: N_(P) and N_(B) are the number of P- and B-pictures remaining inthe current group of pictures, BR_(target) is a subsequently determinedtarget bit rate, and S is one of S_(I), S_(P) or S_(B) according to thepicture type of the previously encoded picture.
 16. A method as claimedin claim 15, further comprising the steps of: obtaining averagequantization step sizes Q_(I), Q_(P), Q_(B) from previously encoded I-,P-, B-pictures, respectively; and determining the target bit allocationT_(I), T_(P), T_(B) for a subsequent respective I-, P-, or B-pictureaccording to:$T_{I} = \frac{R}{1 + \frac{N_{P}X_{P}}{X_{I}K_{P}} + \frac{N_{B}X_{B}}{X_{I}K_{B}}}$$T_{P} = \frac{R}{N_{P} + \frac{N_{B}K_{P}X_{B}}{K_{B}X_{P}}}$$T_{B} = \frac{R}{N_{B} + \frac{N_{P}K_{B}X_{P}}{K_{P}X_{B}}}$

where: X_(I)=S_(I)Q_(I), X_(P)=S_(P)Q_(P), X_(B)=S_(B),Q_(B), K_(P) andK_(B) are constants.
 17. A method as claimed in claim 16, wherein thetarget bit allocations T_(I), T_(P), T_(B) have a lower limit T_(min)determined according to:$T_{\min} = \frac{{BR}_{target}}{K_{3} \times {picture\_ rate}}$

where K₃ is a constant.
 18. A method as claimed in any one of claims 1to 17, wherein the target bit allocation is used to encode an inputpicture, comprising the steps of: deriving macroblocks from the inputpicture; calculating a quantization step-size for each said macroblockaccording to the target bit allocation and its utilization; performingmotion compensation if necessary to each macroblock; frequencytransforming each said macroblock or motion compensated macroblock;quantizing each transformed macroblock using the calculated quantizationstep-size; and coding the quantized macroblock.
 19. A method as claimedin claim 18, wherein the target quantization step-size is used as alower limit for the determined quantization step-size.
 20. A method asclaimed in claim 18 or 19, wherein the determined quantization step-sizefor each macroblock is modulated by a corresponding macroblock activitymasking factor, and the quantization is performed using the modulatedquantization step-size.
 21. A control apparatus for a single pass movingpictures encoder, comprising: an input for receiving a target pictureencoding quality; an input for receiving upper and lower bit ratelimits; and a controller for controlling the encoder so as to encode atleast one picture according to a target bit rate within the upper andlower bit rate limits, predict a current bit rate and an encodingquality based on the result of the encoding, compare the encodingquality of the at least one encoded picture with the target pictureencoding quality, and adjust the target bit rate according to the resultof said comparison and the predicted current bit rate for encodingsubsequent pictures.
 22. A control apparatus as claimed in claim 21,wherein encoding the at least one picture based on the target bit rateincludes determining a target bit allocation for the at least onepicture using the target bit rate.
 23. A control apparatus as claimed inclaim 22, wherein the pictures to be encoded are arranged in groups ofpictures comprising an I-picture and at least one P-picture and/orB-picture, and wherein the controller adjusts the target bit allocationfor each picture or plurality of pictures in each group of pictures. 24.A control apparatus as claimed in claim 21, 22 or 23, wherein thepicture encoding quality is based on encoding quantization step-size.25. A control apparatus as claimed in claim 21, 22 or 23, wherein thepicture encoding quality is based on a mean square error.
 26. A controlapparatus as claimed in claim 21, 22 or 23, wherein the picture encodingquality is based on a signal-to-noise ratio.
 27. A control apparatus asclaimed in claim 25 or 26, wherein the controller includes a localpicture decoder and a comparator for measuring a difference in pictureencoding quality between corresponding input and locally decodedpictures.
 28. A control apparatus as claimed in claim 27, wherein thecontroller includes a picture activity processor for measuring anaverage picture activity for the moving pictures, and wherein thecontroller modifies the measured difference in picture encoding qualityon the basis of the average picture activity.
 29. A control apparatus asclaimed in claim 28, wherein the controller includes a signal-to-noiseprocessor for measuring a signal-to-noise ratio for at least onepreviously encoded picture, and a bit rate predictor for predicting thecurrent bit rate, and wherein the controller adjusts said target bitrate based on the predicted current bit rate and on a difference betweenthe measured signal-to-noise ratio and a target signal-to-noise ratio assaid target picture encoding quality.
 30. A control apparatus as claimedin claim 21, wherein the moving picture encoder includes a frequencytransform coefficient quantizer for quantization of the encoded picturedata, and wherein the controller comprises a bit rate controller coupledto control the quantization step size of the quantizer, a quantizationstep size comparator for comparing, as a measure of encoding quality, anactual quantization step size with a target quantization step size basedon the target picture encoding quality, a bit allocation processorcoupled to control the bit rate controller according to a number of bitsremaining for encoding a group of pictures, and a target bit rateestimator coupled to receive the upper and lower bit rate limits andcoupled to the bit allocation processor and the bit rate controller forcontrolling the quantization so that the required bit rate for thequantized picture data is within the upper and lower bit rate limits.31. A single pass variable bit rate video picture encoder comprising: apicture input for receiving data for a plurality of moving pictures; atarget quality input for receiving a target quality measure for encodedpictures; an encoder output for supplying encoded picture data; a bitrate limit input for receiving upper and lower bit rate limits for theencoded picture data; a bit rate predictor for predicting a current bitrate; a frequency transform processor for frequency transform encodingpicture data from the picture input; a coefficient quantizer forquantizing the encoded picture data according to a quantization stepsize; an encoding quality estimator for measuring an encoding quality ofquantized encoded pictures; and a bit rate controller for dynamicallycontrolling the quantization step size of the coefficient quantizerbased on the predicted current bit rate and a comparison of the targetquality and the measured quality so that the encoder output remainswithin the upper and lower bit rate limits.
 32. A video picture encoderas claimed in claim 31, wherein the pictures to be encoded are arrangedin groups of pictures comprising an I-picture and at least one P-pictureand/or B-picture, the bit rate controller including a frame bit counterfor a number of remaining bits available for encoding a group ofpictures, and a bit allocation processor for controlling the bit ratecontroller according to the remaining available bits so as to adjust atarget bit allocation for each picture or plurality of pictures in eachgroup of pictures.
 33. A video picture encoder as claimed in claim 31 or32, including a quality comparator for comparing the target quality withthe measured encoding quality, and wherein the bit rate controllercontrols the quantization step size based on a difference between thetarget and measured qualities.
 34. A video picture encoder as claimed inclaim 32, including a quality comparator for comparing the targetquality with the measured encoding quality wherein the bit ratecontroller adjusts the target bit allocation to achieve a target bitrate for the picture or plurality of pictures in each group of pictureswhich is determined on the basis of an output from said qualitycomparator.
 35. A video picture encoder as claimed in claim 31, 32, 33or 34, wherein the measured encoding quality is based on thequantization step size.
 36. A video picture encoder as claimed in claim33, including a local decoder for decoding the quantized encoded picturedata, and a quality measurement processor for determining a qualitydifference between corresponding input and locally decoded pictures. 37.A video picture encoder as claimed in claim 36, wherein the qualitymeasurement processor determines a difference in signal-to-noise ratio.38. A video picture encoder as claimed in claim 36, wherein the qualitymeasurement processor determines a mean square error.
 39. A videopicture encoder as claimed in claim 36, 37 or 38, including a pictureactivity processor for measuring an average picture activity for themoving pictures, and wherein the quality measurement processor modifiesthe measured difference in picture encoding quality on the basis of theaverage picture activity.
 40. A video picture encoder as claimed inclaim 37, including a signal-to-noise processor for measuring asignal-to-noise ratio for at least one previously encoded picture, and abit rate predictor for estimating a current bit rate, and wherein thebit rate controller adjusts said target bit allocation based on theestimated current bit rate and on a difference between the measuredsignal-to-noise ratio and a target signal-to-noise ratio as said targetpicture encoding quality.
 41. A video picture encoder as claimed inclaim 31, including a quantization step size comparator for comparing,as a measure of encoding quality, an actual quantization step size witha target quantization step size based on the target picture encodingquality, a bit allocation processor coupled to control the bit ratecontroller according to a number of bits remaining for encoding a groupof pictures, and a target bit rate estimator coupled to receive theupper and lower bit rate limits and coupled to the bit allocationprocessor and the bit rate controller for controlling the quantizationso that the required bit rate for the quantized picture data is withinthe upper and lower bit rate limits.